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Evaluating the Accuracy of Emergency Medicine Resident 
Interpretations of Abdominal CTs in Patients with Non-Traumatic 
Abdominal Pain 

Abdominal computed tomography (CT) is widely used as a diagnostic tool in emergency 
medicine (EM) to accurately diagnose abdominal pain. EM residents must be able to offer 
preliminary interpretations of CT imaging. In this study, we evaluated the preliminary 
interpretation ability of a sample of emergency residents presented with adult abdominal 
CT images, and compared their results with those of radiology residents. We conducted a 
prospective observational study from November 16, 2008 to June 30, 2009. During this 
time, we gathered preliminary interpretations of consecutive abdominal CT made by 
emergency and radiology residents. We assessed the discrepancy rates of both samples by 
comparing their findings to the final reports from attending radiologists. A total of 884 
cases were enrolled in the present study. The discrepancy rates of emergency and radiology 
residents were 16.7% and 12.2%, respectively. When female genital organs, peritoneum, 
adrenal glands, or the musculoskeletal system were abnormal, we found that emergency 
residents' preliminary interpretations of CT images were insufficient compared to those of 
radiology residents. Therefore more formal education is needed to emergency residents. If 
possible, the preliminary interpretations of radiology attending physicians are ideal until 
improving the ability of interpretations of emergency residents in abdomen CT. 
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INTRODUCTION 

Acute abdominal pain is a common chief complaint in clinical 
practice, particularly in the emergency department (ED) (1). In 
the ED, prompt differential diagnosis of acute abdominal pain 
is essential, since delayed or missed diagnoses are associated 
with high morbidity and mortality (1-4). In order to obtain an 
accurate diagnosis, abdominal computed tomography (CT) 
has been widely used as a diagnostic tool (1-4). Therefore, the 
ability offer accurate preliminary interpretations of CT images 
is required for an emergency resident (5). 

The ability of radiology residents to interpret radiologic im- 
ages, including CT images, has been well established in previ- 
ous studies (6-15). However, only a few such studies have been 
conducted for clinical department residents, focusing on sim- 
ple radiograph or CT images of traumatic patients (5, 16). There- 
fore, in this study, we evaluated the ability of emergency resi- 
dents to accurately interpret abdominal CT images for non-trau- 
matic patients. 

We conducted a prospective observational study to investi- 
gate the discrepancy rates of preliminary reports of emergency 



abdominal CT imaging of emergency residents and to compare 
them to the results of similar studies of radiology residents. We 
also analyzed the factors-including major abnormal organs, 
time of interpretation, and grades of residents that affect the 
accuracy of emergency resident reports. To evaluate the rela- 
tionship between discrepancies and clinical outcomes, we com- 
pared ED stay time, admission rate, admission length, opera- 
tion rate, revisit rate, and mortality rate of patients whose ab- 
dominal CT were inaccurately preliminary interpreted with pa- 
tients whose CT were accurately interpreted. 

MATERIALS AND METHODS 
Subjects and study period 

We enrolled all CT scan protocols involving the entire abdomen 
and pelvic cavity, e.g., abdominal pelvic scan, liver scan, biliary- 
gall bladder scan, pancreas scan, abdominal CT angiography, 
that were ordered by the ED at the Samsung Medical Center, a 
tertiary educational hospital, from November 16, 2008 to lune 
30, 2009. A Centricity Web-Fax system (G.E., Fairfield, CT, USA) 
was used for all radiologic studies. Exclusion criteria included 
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trauma patients, pediatric patients (under 15 yr), and patients 
whose chief complaint was not abdominal pain. Cases where 
first preliminary interpretations were performed by attending 
faculty in emergency, radiology, or radiology residents were ex- 
cluded. Cases where the primary pathognomonic findings on 
the final formal report were not for abdominal organs, but for 
lungs that were visualized by the abdominal CT, were excluded. 

Data collection 

In order to collect preliminary interpretation results from emer- 
gency and radiology residents, data collection sheets were cre- 
ated. First, emergency residents recorded their preliminary in- 
terpretations on the data collection sheets. Second, the residents 
requested and received preliminary interpretation results from 
the radiology department by telephone, and entered this infor- 
mation on the data collection sheets. The grades assigned by 
emergency and radiology residents who performed the prelim- 
inary interpretations, and the time of the interpretation, were 
also collected. The time of interpretation was classified as either 
on-duty time (8 a.m. and 5 p.m. Monday to Friday) or off-duty 
time. 

After requesting and receiving preliminary interpretations 
from radiology, the emergency residents supplied patient infor- 
mation, including chief complaint and laboratory findings, to 
the radiology residents. Discussions about the CT images were 
not permitted. 

Abnormal organ classification and discrepancies 

In this study, the final formal reports that were interpreted by 
radiology attending physicians were used as the gold standard 
to interpret the CT scans. The abnormal organs identified by the 
final formal reports were classified into nine categories: liver 
and spleen, gall bladder and biliary tract, pancreas, hollow vis- 
cera except appendix, appendix, urinary system (kidney, ureter, 
bladder), female genital organ (uterus, ovary, pelvic cavity), 
blood vessels, and miscellaneous (peritoneum, adrenal gland, 
musculoskeletal system). If there were more than two abnormal 
organs in the final formal report, only one organ was selected 
based on the most pathognomonic findings of the report. If 
there were no radiological abnormal findings on the final for- 
mal reports, the abnormal organs were classified as non-specif- 
ic abdominal pain. 

A discrepancy was defined as a change in diagnosis, treat- 
ment, or disposition. Minor differences between final formal 
reports and preliminary interpretations that did not result in a 
change were not considered discrepancies. Discrepancies were 
classified into three categories, with the first being false nega- 
tive in which the most important diagnosis in the final reports 
was not made in preliminary reports. The second category rep- 
resented a misdiagnosis, in which preliminary interpretations 
and final interpretations described the same organs, but yield- 



ed different diagnoses. The final category was false positive, in 
which the final reports were of normal findings, but abnormal 
findings were recorded in the preHrnfnary reports. 

To classify abnormal organs and resolve discrepancies, two 
emergency medicine faculty members independently com- 
pared the final formal reports with preliminary interpretations 
from emergency medicine and radiology residents. If the opin- 
ions of both of the faculty members did not coincide, the opin- 
ions were discussed until an agreement was reached. 

Clinical outcomes of patients 

Patient outcomes were evaluated by retrospective chart review. 
We investigated each patient's ED stay time and whether or not 
the patient was admitted. If the patient was admitted, his or her 
admission length, whether or not he or she underwent an emer- 
gency operation, and whether or not he or she survived after 
discharge, were investigated. If the patient was not admitted, 
we investigated whether or not the same patient revisited the 
ED within 24 hr with the same chief complaint. 

Statistical analysis 

Statistical analyses were performed using STATA 11.0 (Stata- 
Corp, Texas, USA). To assess the differences between the dis- 
crepancy rates of emergency and radiology residents, we per- 
formed chi-square tests with P values < 0.05 considered to be 
statistically significant. To determine the factors-abnormal or- 
gans, grade of residents, time of interpretations-that affect the 
accuracy of CT interpretations by emergency residents, multi- 
variate analyses using a logistic regression test model were ap- 
plied with robust variance estimates across the clusters of the 
emergency residents' individual identification. The regression 
model was adjusted with patient's sex, age, whether admitted 
or not, mortality and emergency department stay time. Hosmer 
and Lomeshow tests were used to evaluate modeling fitness. 

To evaluate the relationship between discrepancies and clin- 
ical outcomes, the admission rate, emergency operation rate, 
revisit rate, and mortality rate were compared by chi-square 
test for correct and incorrect interpretations by emergency resi- 
dents. Admission length and ED stay time were compared by 
Student's t-test. 

Ethics statement 

This study was approved by the institutional review board of the 
Samsung Medical Center (IRB File No. 2010-01-091). The in- 
formed consent from the patients and residents who partici- 
pated in this study was waived by the board. 

RESULTS 

A total of 1,153 abdominal CT scan examinations were ordered 
by the ED during the study period. We excluded 140 cases be- 
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cause they were trauma patients or pediatric patients, and 110 
cases because the chief complaint was not abdominal pain. Eight 
cases were excluded because the first preliminary interpreta- 
tions were performed by emergency medicine faculty mem- 
bers. Two cases in which data regarding preliminary interpreta- 
tions were not complete and nine cases in which the primary 
pathognomonic organ was the lung were excluded. After exclu- 
sions, 884 cases were enrolled. 

In 884 cases, 741 cases were examined by the abdominal pel- 
vic protocol, 14 by liver protocol, 109 by biliary gall bladder pro- 
tocol, 11 by pancreas protocol, and 9 by abdominal CT angiog- 
raphy. The mean age of the enrolled patients was 46.83 yr; 493 
(55.8%) patients were female, and 391 (44.2%) patients were 
male (Table 1). 

In the ED, 404 (45.7%) scans were preUminarily interpreted 
by first year residents, 203 (22.9%) by second year residents, 23 1 
(26.1%) by third year residents, and 46 (5.2%) by fourth year 
residents. In the radiology department, 421 (47.6%) scans were 

Table 1 . Baseline characteristics of 884 enrolled patients 
Characteristic Value 

Age(yr) 48.83 + 18.50 

Sex (%) 

Male 

Female 
ED stay time (hr) 
ED revisit within 48 hr (%) 
Admission (%) 
Emergency operation (%) 
Mortality (%) 

ED, Emergency department. 



preliminarily interpreted by second year residents and 463 
(52.4%) by third year residents. 

Discrepancy rates 

In emergency resident preliminary reports, there were 148 dis- 
crepant cases (16.7%). In radiology resident preliminary reports, 
there were 108 discrepant cases (12.2%) (Table 2). The discrep- 
ancy rate of emergency residents was significandy higher than 
that of radiology residents (P = 0.01). The discrepancy rate of 
emergency residents was significandy higher than that of radi- 
ology residents for images of female genital organs (P < 0.01) 
and miscellaneous organs {P = 0.01) (Fig. 1). 

Factors affecting the accuracy of emergency resident CT 
interpretations 

In the regression test used to analyze the factors that affected 
the discrepancy, the grades of residents and times of interpreta- 
tions did not affect discrepancy rates. Except hollow viscera, all 
other cases of classified abnormal organs were associated with 
significant discrepancies when compared to cases of non-spe- 



Table 2. The discrepancy rates of emergency and radiology residents 

391 (44.23) EM residents Radiology residents 

493(5576) Total discrepancies (%) 148(167) 108(12.2) 

1 4.1 2 + 21 .00 Fa | ge negative report (%) -| 35 pi 2 ) 97 (89.8) 

64 < 7 - 24 ) False positive report (%) 1 1 (7.4) 9 (8.3) 

336(38.01) Misinterpretation 2(1.3) 2(1.8) 
1 42 (1 6.06) of pathology presents (%) 

1 1 (1 .25) No discrepancies (%) 736 (83.3) 776 (87.8) 
EM, Emergency medicine. 




Abnormal organ classification 



Fig. 1. A comparison of the discrepancy rate between emergency medicine and radiology residents according to the abnormal organ. 'There were statistically significant differ- 
ences for female genital organs and miscellaneous organs. ER, Emergency resident; RA, Radiology resident; Uterus adnexa, Female genital organs (uterus, adnexa, ovary, pelvic 
cavity); Hollow viscera, Hollow viscera except appendix; Miscellaneous, Peritoneum, adrenal gland, musculoskeletal system; NSAP, Non-specific abdominal pain (no radiological 
abnormality). 
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Table 3. The results of multivariate logistic regression tests of discrepancy after ad- 
justed patient's factors* 



Factors 


Odds ratio 
(95% Confidence Interval) 


P value 
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D 87 10 56-1 321 
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0.55 


Third 
1 f III U 


I . I 0 ^U.OD- I .00) 




0.33 


Fourth 


u.oy yj.c i - 1 .0/ j 




0.32 


Time of interpretation 








On-duty time 


Reference 






Off-duty time 


1 .25 (U.oU-1 .95) 




0.32 


The classified abnormal organs 








NSAP (normal findings) 


Reference 






Liver, spleen 


4.87 (1.76-13.44) 


< 


0.01 


Biliary tract 


7.52 (3.56-15.87) 


< 


0.01 


Pancreas 


11.47 (4.53-29.04) 


< 


0.01 


Hollow viscera 


2.27 (0.92-5.58) 




0.07 


Appendix 


2.90 (1.19-7.04) 




0.02 


Urinary system 


2.69(1.51-4.81) 


< 


0.01 


Female genital system 


12.50 (6.59-23.73) 


< 


0.01 


Blood vessels 


8.14(2.39-27.76) 


< 


0.01 


Miscellaneous 


4.14(1.34-12.76) 




0.01 



The regression model was adjusted with patient's sex, age, whether admitted or not, 
mortality and emergency department stay time. On-duty time: 8 a.m. and 5 p.m. 
Monday to Friday. Off-duty time: any time other than on-duty time. NSAP: Non-specif- 
ic abdominal pain (No radiologic abnormality); Hollow viscera: Hollow viscera except 
appendix; Female genital organs: Uterus, adnexa, ovary, pelvic cavity; Miscellaneous: 
Peritoneum, adrenal gland, musculoskeletal system. 

cific abdominal pain (Table 3). 

The relationship between clinical outcomes and accuracy 
of emergency resident interpretations 

There were no significant differences in admission rates be- 
tween inaccurately diagnosed patients and accurately diag- 
nosed patients (37.3% vs 41.2%, respectively, P = 0.37) or ED 
stay time (14.3 hr vs 13.1 hr, respectively, P = 0.55). There were 
no significant differences in admission length (9.2 days vs 11.0 
days, respectively, P = 0.36), emergency operation rate (43.2% 
vs 34.6%, respectively, P = 0.20), mortality rate (1.3% vs 0.7%, 
respectively, P = 0.49) of the admitted patients, and ED revisit 
rate (5.6% vs 5.7%, respectively, P = 0.96) of the discharged pa- 
tients. 

DISCUSSION 

We investigated and compared discrepancy rates of abdominal 
CT interpretations made by emergency residents and radiology 
residents. Several previous studies have assessed discrepancy 
rates of radiology resident interpretations of CT scans (5-8, 10, 
12, 13, 15, 17-19). In these studies, the discrepancy rate varied 
from 1.2% to 10% (5-7, 10, 12, 13, 15, 18). In the present study, the 
discrepancy rates of emergency and radiology residents were 
16.7% and 12.2%, respectively. These values were higher than 
those reported in previous studies. However, precise compari- 
sons between studies are not possible because of many differ- 



ences, including resident specially, the part of the body exam- 
ined by CT scans, and the disease scope that was enrolled. The 
criteria for defining discrepancies were also different because 
there are no standard definitions of discrepancy. In previous 
studies, both major and minor discrepancies were assessed, 
with definitions of "major" and "minor" differing between stud- 
ies (5-7, 10, 12, 13, 18). In the present study, discrepancies were 
recognized when they resulted in a change in diagnosis, treat- 
ment, or disposition, leading to high discrepancy rates. 

In the present study, the discrepancy rate of emergency resi- 
dents (16.7%) was higher than that of radiology residents (12.2%) 
when the same CT scans and criteria were used. This may be 
explained by the different experiences and levels of education 
of the residents included in the study regarding CT interpreta- 
tion. At our institute, radiology residents are trained systemati- 
cally in radiology of the abdomen, chest, and musculoskeletal 
system during the first year. In addition the residents had par- 
ticipated in teaching conferences before their emergency de- 
partment rotation. In Samsung Medical Center, radiology resi- 
dents are on call during their 2nd, 3rd years of radiology train- 
ing. On the other hand emergency residents are only trained for 
a few months in an informal training program, and separately 
focused on common and serious diseases. The residents had 
participated in teaching conferences in emergency radiology 
before their emergency department rotation. Usually this train- 
ing program has conducted during 1st year of emergency resi- 
dent training. In addition all emergency residents attend a radi- 
ology conference once a week. This training difference may ex- 
plain why CT scans of female genital organs and miscellaneous 
organs (the peritoneum, adrenal gland and musculoskeletal 
system) were more likely to be misinterpreted by emergency 
residents, as these organs are difficult to interpret in general, 
and are often the source of errors in interpretation (20). Abnor- 
malities of these organs are relatively uncommon compared to 
other organs (21). The discrepancy rates for the appendix and 
biliary track, which are the most common regions of abdominal 
pain in the ED, were similar between ED and radiology residents. 
Therefore we could suggest that education of emergency resi- 
dents is important to decrease the discrepancy rates. Especially 
the education for unfamiliar organs that were more likely to be 
misinterpreted by emergency residents may be needed to in- 
crease the capacity of interpretation of unfamiliar organs in emer- 
gency residents. And this type of education may be able to de- 
crease the discrepancy rates of emergency residents. In the fu- 
ture, we need further study of the relationship between formal 
radiology education of emergency residents and the discrepan- 
cy rates. 

Tieng et al. (7) reported that the discrepancy of radiology res- 
idents were common. And they emphasized that ED physicians 
should exercise caution when relying on radiology residents' 
interpretation of body CT scans. We found that the discrepancy 
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of emergency residents were higher than that of radiology resi- 
dents. Therefore emergency residents may need to discuss with 
radiology physicians about their preliminary interpretation of 
body CT scans when emergency residents make decisions based 
on their interpretation of body CT scans. If possible, the prelim- 
inary interpretations of radiology physicians are needed to make 
up for the interpretation of emergency residents. Some studies 
(22, 23) reported that the importance of systematic support to 
reduce the discrepancy in radiology imaging. Therefore we think 
that we could reduce the discrepancy of emergency residents 
by systematic support. More investigations are required for eval- 
uating the effect of systematic supports on the discrepancy. 

Previous studies showed that discrepancy rates were associ- 
ated with the training level of the residents (6, 18). The discrep- 
ancy rates of senior residents are lower than those of junior res- 
idents (6, 18). In the present study, we detected no significant 
difference in discrepancy rates associated with level of training. 
We did additional analysis between the training level of the res- 
idents and the disease severity. There were no significant differ- 
ence between the training level of the residents and the disease 
severity; the admission rate, the emergency operation rate and 
the mortality rate. Therefore this may be because the imaging 
education program for emergency residents is concentrated 
during the first few months of the first year of residency for emer- 
gency physicians at our institute. Also lack of a formal educa- 
tion about radiology contributed to induce this result. 

It is difficult to assess whether discrepancies in preliminary 
interpretation affect clinical outcomes, since there is no stan- 
dard method for making such comparisons. As a result, various 
methods have been used in previous studies (9, 12, 18, 24). In 
this study, the same parameters were used; e.g., ED stay time, 
admission rate, admission length, emergency operation rate, 
ED revisit rate, and mortality rate. There were no significant dif- 
ferences for these parameters between patients who were diag- 
nosed accurately or inaccurately by emergency residents. Our 
data of clinical effect are consistent with other study (7, 11, 12, 
23). It seems that EM residents are relying on other available 
data (patient's history, physical examination, observation and 
consultation) to prevent clinical mistakes. However the inter- 
pretation of that result was very limited because the discussion 
with radiology residents or notification to ED faculties after the 
preliminary interpretation data collection was not restricted 
due to ethical problems. 

This study had several limitations. First, the composition of 
the patient sample, and the educational levels of the residents, 
varied due to national origin, regional origin, and institutes. Al- 
though we adjusted the results for individual identifications of 
emergency residents, variation in individual ability and experi- 
ence could have affected the results. Therefore, the results of 
this study cannot be generalized. The results of this study may 
be more appropriate for use as basic data to establish CT imag- 



ing education guidelines for emergency residents. Second, like 
similar previous studies, we considered final reports of radiolo- 
gy faculty members to represent the gold standard. However, a 
previous study showed that 2.3% of major body CT scans were 
incorrectly interpreted by attending radiology faculty members 
(24). Third, because radiology residents started ED rotation at 
2nd year of resident, we could not compare the same training 
period between emergency residents and radiology residents. 
Finally, we were unable to clearly evaluate the influence of dis- 
crepant interpretations on clinical outcomes due to ethical prob- 
lems. 

Emergency residents were more likely to inaccurately inter- 
pret preliminary CT scans than radiology residents. When the 
female genital organs, peritoneum, adrenal glands, or the mus- 
culoskeletal system were abnormal, the preliminary interpreta- 
tion abilities of emergency residents proved to be especially in- 
sufficient compared to those of radiology residents. Therefore 
more formal education is needed to emergency residents. EM 
physicians should pay attention to clinical decision when rely- 
ing on emergency residents' interpretation of body CT. If possi- 
ble, the preliminary interpretations of radiology attending phy- 
sicians are ideal until improving the ability of interpretations of 
emergency residents in abdomen CT. 
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